Characterizing Deep Gaussian Processes via Nonlinear Recurrence Systems

Recent advances in Deep Gaussian Processes (DGPs) show the potential to have more expressive representation than that of traditional Gaussian Processes (GPs). However, there exists a pathology of deep Gaussian processes that their learning capacities reduce significantly when the number of layers increases. In this paper, we present a new analysis in DGPs by studying its corresponding nonlinear dynamic systems to explain the issue. Existing work reports the pathology for the squared exponential kernel function. We extend our investigation to four types of common stationary kernel functions. The recurrence relations between layers are analytically derived, providing a tighter bound and the rate of convergence of the dynamic systems. We demonstrate our finding with a number of experimental results.Comment: AAAI 202

PROBABILISTIC MODEL DISCOVERY RELATIONAL LEARNING AND SCALABLE INFERENCE

Department of Computer Science and EngineeringThis thesis studies interesting problems in compositionality for machine learning models under some settings including relational learning, scalability and deep models. Compositionality is the terminology describing the process of building small objects to complex ones. Bringing this concept into machine learning is important because it appears in many aspects from infinitesimal atomic to planetary structures. In this thesis, machine learning models center around Gaussian process of which covariance function is compositionally constructed. The proposed approach builds methods that can explore compositional model space automatically and efficiently as well as strives to address the interpretability for obtained models. The aforementioned problems are both important and challenging. Considering multivariate or relational learning is de facto in time series analysis for many domains. However, the existing methods of compositional learning are inapplicable to extend to such a setting since the explosion in model space makes it infeasible to use. Learning compositional structures is already a time-consuming task. Although there are existing approximation methods, they do not work well for compositional covariances. This makes it even harder to propose a scalable approach without sacrificing model performances. Finally, analyzing hierarchical deep Gaussian processes is notoriously difficult especially when incorporating different covariance functions. Previous work focuses on a single case of covariance function and is difficult to generalize for many other cases. The goal of this thesis is to propose solutions to the given problems. The first contribution of this thesis is a general framework for modeling multiple time series which provides descriptive relations between time series. Second, this thesis presents efficient probabilistic approaches to address the model search problem which previously is done by exhaustive enumerating evaluation. Furthermore, a scalable inference for Gaussian process is proposed, providing accurate approximation with guarantees of error bounds. Last but not least, to address the existing issues in deep Gaussian process, this thesis presents a unified theoretical framework to explain the pathology in deep Gasssian processes with better error bounds for various kernels compared to existing work and rates of convergence.ope

Conditional Support Alignment for Domain Adaptation with Label Shift

Unsupervised domain adaptation (UDA) refers to a domain adaptation framework in which a learning model is trained based on the labeled samples on the source domain and unlabelled ones in the target domain. The dominant existing methods in the field that rely on the classical covariate shift assumption to learn domain-invariant feature representation have yielded suboptimal performance under the label distribution shift between source and target domains. In this paper, we propose a novel conditional adversarial support alignment (CASA) whose aim is to minimize the conditional symmetric support divergence between the source's and target domain's feature representation distributions, aiming at a more helpful representation for the classification task. We also introduce a novel theoretical target risk bound, which justifies the merits of aligning the supports of conditional feature distributions compared to the existing marginal support alignment approach in the UDA settings. We then provide a complete training process for learning in which the objective optimization functions are precisely based on the proposed target risk bound. Our empirical results demonstrate that CASA outperforms other state-of-the-art methods on different UDA benchmark tasks under label shift conditions

Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior

Choosing a proper set of kernel functions is an important problem in learning Gaussian Process (GP) models since each kernel structure has different model complexity and data fitness. Recently, automatic kernel composition methods provide not only accurate prediction but also attractive interpretability through search-based methods. However, existing methods suffer from slow kernel composition learning. To tackle large-scaled data, we propose a new sparse approximate posterior for GPs, MultiSVGP, constructed from groups of inducing points associated with individual additive kernels in compositional kernels. We demonstrate that this approximation provides a better fit to learn compositional kernels given empirical observations. We also provide theoretically justification on error bound when compared to the traditional sparse GP. In contrast to the search-based approach, we present a novel probabilistic algorithm to learn a kernel composition by handling the sparsity in the kernel selection with Horseshoe prior. We demonstrate that our model can capture characteristics of time series with significant reductions in computational time and have competitive regression performance on real-world data sets.Comment: AAAI 202

Three essays on the trade effect of US farm subsidies

The thesis comprises three essays investigating the effect of US farm subsidies on its international trade. The results, after accounted for potential endogeneity, indicate that US farm subsidies have significant on US exports, and imports at both aggregate and disaggregate level. The finding would have important implications for policy makers and trade negotiators in the context that reform in agricultural sector is the central debate in Doha Rounds among WTO members

Modélisation micromécanique des couplages hydromécaniques et des mécanismes d'érosion interne dans les ouvrages hydrauliques

Les matériaux granulaires multiphasiques occupent une place très importante dans notre environnement qui suscitent un grand intérêt de nombreuses communautés scientifiques, notamment celles de la mécanique des sols ou de la géotechnique. Le caractère divisé permet aux milieux granulaires multiphasiques d'avoir un comportement mécanique global qui trouve leur origine, leur distribution et interactions entre les phases de composition. Un modèle de couplage hydromécanique est présenté dans ce travail de thèse pour l'application à la modélisation microscopique des couplages hydromécaniques dans les matériaux granulaires saturés. Le modèle numérique est basé sur un couplage de la méthode des éléments discrets (DEM) avec une formulation en volumes finis, à l'échelle des pores (PFV), du problème de l'écoulement d'un fluide visqueux incompressible. Le solide est modélisé comme un arrangement de particules sphériques avec des interactions de type élasto-plastique aux contacts solide-solide. On considère un écoulement de Stokes incompressible, en supposant que les forces inertielles sont négligeables par rapport aux forces visqueuses. La géométrie des pores et leur connectivité sont définies sur la base d'une triangulation régulière des sphères, qui aboutit à un maillage tétraédrique. La définition des conductivités hydrauliques à l'échelle des pores est un point clef du modèle, qui se rapproche sur ce point des modèles de type \textit{réseau poral}. Une importance particulière réside dans les lois d'interactions fluide-solide permettant de déterminer des forces de fluide appliquées sur chacune des particules, tout en assurant un coût de calcul acceptable pour la modélisation en trois dimensions avec plusieurs milliers de particules. Des mesures de perméabilités sur des assemblages bi-disperses de billes de verre sont présentées et comparées aux prédictions du modèle et aux formules empiriques et semi-empiriques dans la littérature, ce qui valide la définition de la conductivité locale et met en évidence le rôle de la distribution granulométrique et la porosité. Une approche numérique pour analyser l'interaction mécanique fluide-solide et les mécanismes d'érosion interne est finalement présentée.Multiphase granular materials occupy a very important place in our environment that are of great interest to many scientific communities, including those of soil mechanics or geotechnical engineering. The divided nature allows multiphase granular media to have a global mechanical behavior which originates from all component phases, their distribution and interactions. A coupled hydromechanical model is presented in this work for the application to microscopic modeling of coupled hydromechanical effects in saturated granular materials. The numerical model uses a combination of the discrete element method (DEM) with a pore-scale finite volume (PFV) formulation of flow problem of an incompressible viscous fluid. The solid is modeled as an assembly of spherical particles, where contact interactions are governed by elasto-plastic relations. Stokes flow is considered, assuming that inertial forces are small in comparison with viscous forces. Pore geometry and pore connections are defined locally through regular triangulation of spheres, from which a tetrahedral mesh arises. The definition of pore-scale hydraulic conductivities is a key aspect of this model. In this sense, the model is similar to a pore-network model. The emphasis of this model is, on one hand the microscopic description of the interaction between phases, with the determination of the forces applied on solid particles by the fluid, on the other hand, the model involves affordable computational costs, that allow the simulation of thousands of particles in three dimensional models. Permeability measurements on bi-dispersed glass beads are reported and compared with model predictions and empirical formulas/semi-empirical in the literature, validating the definition of local conductivities and bringing out the role of particle size distribution and porosity. A numerical approach to analyze the fluid-solid mechanical interaction and mechanisms of internal erosion is finally presented